Skip to content

LUCENE-14611: Reduce memory pressure in TestTermInSetQuery.testDuel#16300

Merged
dweiss merged 6 commits into
apache:mainfrom
lsh1215:lucene-14611-testduel-oom
Jul 8, 2026
Merged

LUCENE-14611: Reduce memory pressure in TestTermInSetQuery.testDuel#16300
dweiss merged 6 commits into
apache:mainfrom
lsh1215:lucene-14611-testduel-oom

Conversation

@lsh1215

@lsh1215 lsh1215 commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

Description

This reduces avoidable memory pressure in TestTermInSetQuery.testDuel.

The test is checking query equivalence, not query-cache behavior. It still uses the randomized test-framework newSearcher(reader), but disables query caching for this test so repeated equivalence checks do not retain cached doc-id sets across iterations.

For the no-score comparison path, the test now collects matching doc IDs with the existing test-framework FixedBitSetCollector and compares the resulting bit sets. This avoids materializing full sorted TopDocs when only doc-id equality matters.

This is intentionally different from prior range-cap attempts such as #15084 and #15118: it does not reduce randomized input sizes or change the stress shape. Instead, it removes the pressure sources that are outside the behavior this test is meant to validate.

Local investigation

I could not reproduce the OOM on my machine with the original issue command, likely due to local environment differences. To make the failure mode visible locally, I used the same stress shape with the test JVM constrained to a 16m heap.

With the original code, that 16m squeeze reproduced:

java.lang.OutOfMemoryError: Java heap space

Heap dump inspection, jstat, and JFR heap profiling pointed to avoidable pressure from the randomized query cache retaining doc-id sets and from the no-score branch materializing sorted TopDocs.

Verification

Ran with JDK 26.

./gradlew tidy
BUILD SUCCESSFUL

./gradlew :lucene:core:compileTestJava
BUILD SUCCESSFUL

./gradlew :lucene:core:test --tests "org.apache.lucene.search.TestTermInSetQuery.testDuel" -Ptests.filter=@Nightly -Ptests.jvms=1 -Ptests.seed=B03A5F38917C1431 -Ptests.useSecurityManager=false -Ptests.gui=true -Ptests.file.encoding=UTF-8 -Ptests.vectorsize=128 -Ptests.forceintegervectors=true
:lucene:core:test (SUCCESS): 1 test

./gradlew :lucene:core:test --tests "org.apache.lucene.search.TestTermInSetQuery" -Ptests.jvms=1 -Ptests.seed=B03A5F38917C1431 -Ptests.useSecurityManager=false -Ptests.gui=true -Ptests.file.encoding=UTF-8 -Ptests.vectorsize=128 -Ptests.forceintegervectors=true
:lucene:core:test (SUCCESS): 15 tests, 1 skipped

./gradlew :lucene:core:test --tests "org.apache.lucene.search.TestTermInSetQuery.testSkipperOptimizationGapAssumption" -Ptests.asserts=true -Ptests.file.encoding=UTF-8 -Ptests.gui=true -Ptests.jvmargs= -Ptests.jvms=4 -Ptests.seed=A8777D0B61FEFFBE -Ptests.vectorsize=128
:lucene:core:test (SUCCESS): 1 test

I also re-ran the local 16m squeeze with the inner testDuel loop temporarily raised from 100 to 10000; with this fix applied, it completed successfully instead of reaching the Java heap OOM state observed with the original code.

Fixes #14611.

@github-actions github-actions Bot added this to the 11.0.0 milestone Jun 27, 2026
@lsh1215
lsh1215 marked this pull request as ready for review June 27, 2026 03:33
@lsh1215

lsh1215 commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

Hi @dweiss, thanks for filing this issue. You reviewed the earlier attempt in #15118, so I thought you might want to take a look at this one.

It takes a different approach from the range-capping in #15118. Instead of shrinking the randomized inputs, it goes after the memory pressure itself. Two changes:

  • Disable query caching during the equivalence check, so cached doc-ids don't stick around across iterations.
  • Use a bitset comparison on the no-score path instead of the materialized TopDocs.

The original stress shape stays the same. I reproduced the OOM under a 16MB heap and it passes after the change.

Let me know what you think when you get a chance.

@dweiss

dweiss commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

This looks great, thank you.

assertEquals(td1.scoreDocs[i].doc, td2.scoreDocs[i].doc);
if (scores) {
if (scores) {
final TopDocs td1 = searcher.search(q1, maxDoc);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For pure topDocs use it would be enough to use numDocs, because deleted documents - we don't care!
Of course this does not matter if the index has no deletions (not sure about this test). I just noticed this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, that's a good point.

@dweiss
dweiss merged commit 6f01543 into apache:main Jul 8, 2026
12 checks passed
dweiss added a commit that referenced this pull request Jul 8, 2026
dweiss added a commit that referenced this pull request Jul 8, 2026
@dweiss dweiss modified the milestones: 11.0.0, 10.6.0 Jul 8, 2026
@dweiss

dweiss commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

I've backported all changes to this test to 10x.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TestTermInSetQuery.testDuel occasionally OOMs

3 participants